the extraordinary battle over what stays secret

Paul Thompson was shot dead while being driven home in a taxi through Belfast in April 1994. He was murdered in a sectarian attack by loyalist paramilitaries who had set out to kill Catholics.

The taxi driver was a friend who was collecting a fare at another address in the Springfield Park area in the west of the city. But no one at the address had rung the taxi firm.

The call is suspected to have been a deliberate decoy by the killers who wanted to lure a taxi as prey. Paul was not specifically targeted. They could not have known he would ask for a lift or be in the area.

One resident described hearing a heavy burst of gunfire after 23:00, running to the car and finding Paul.

Members of a banned loyalist terror group, the Ulster Defence Association, using the cover name Ulster Freedom Fighters, claimed responsibility for the killing. During the Troubles in Northern Ireland, the group repeatedly murdered Catholic civilians like Paul.

On the morning of the day that Paul was killed, locals had told police about a hole in a barrier – known as the ‘Peace Line’ – which separated the largely Catholic Springfield Park area from a largely Protestant neighbouring area. The residents were concerned the breach could give loyalist paramilitaries access to launch an attack.

The hole was not fixed. After the murder, those responsible escaped through it.

An inquest into Paul’s death was opened in 1995 but never concluded. No-one has ever been held to account. There has never been a full investigation.

Left by themselves to fight for answers, his mother Margaret and brother Eugene, both now dead, came to suspect collusion between British security forces and informants – meaning state agents – in the group responsible.

In 2024, a coroner decided to release to the family a summary of secret information held by the police. This was said by the coroner to be of central relevance to the case and the police supported the move. But the British government and MI5 argued this would prejudice national security and mounted a legal challenge.

The case, which centred on who had the power to decide to release the information, reached the Supreme Court last summer, with Paul Thompson’s brother, Eugene, tuning in to hearings from a hospice, as he neared death from cancer.

The case is widely viewed as a major test of the state’s policy of neither conforming nor denying (NCND) information involving national security.

The NCND policy is a position adopted by the government and security services when responding to questions about sensitive matters. It means there will be no confirmation, no denial, and no acknowledgement that the information requested is true, false, or even exists.

Eugene Thompson’s barrister, Monye Anyadike-Danes KC, said the family had concluded the murder happened because the authorities were colluding with the “very group that actually perpetrated the killing”.

She said Eugene was “trying to find out why his younger brother was murdered” and felt a “deep responsibility”.

Eugene, she added, believed the public should be able to trust that those who kill will be held accountable, especially when they are agents of the state. She asked the judges not to let the NCND policy be used to “conceal the involvement of state parties in the killing of an ordinary citizen”.

Just before Christmas, the court ruled that the government’s assessment of damage to national security “should have been accepted by the coroner” unless it was plainly irrational or there was no supporting evidence.

The justices found the coroner had failed to understand it was “solely” for the government to make submissions on national security, not the chief constable of the Police Service of Northern Ireland (PSNI), Jon Boutcher, who had argued for the release of the summary of police files.

Paul Thompson’s case is part of a battle over the truth and whether it can be told. The battle concerns official secrets, often connected to state agents, and it pits bereaved families and senior police officers against the government and the security service, MI5.

It raises a profound question: can the state, especially when it is implicated in killing, be trusted as the arbiter of what should remain secret?

Other cases in the coming months will force these issues further into the spotlight – and they come at a time when MI5’s integrity is under significant scrutiny.

Yet the Supreme Court’s ruling in the Thompson case creates a precedent that says the government and MI5 can withhold information from bereaved families, for national security reasons, even when the police want it to be disclosed.

Grainne Teggart, from Amnesty International, described it as a “grim day for truth” and part of a pattern of “national security being used as a shield to cover up and conceal the state’s role in very serious crimes such as murder and other serious human rights violations.”

The government, by contrast, welcomed the court’s “unanimous judgment”.

“This is a highly complex case with wide-ranging implications,” it said. “The government will therefore take time to fully consider all aspects of this judgment.”

During the Troubles, the state made widespread use of informants, otherwise known as agents, who were tasked with carrying out actions and often paid.

Agents existed in the IRA and in loyalist paramilitary groups. They were run by the police, the army, and MI5.

The former Met Police chief Lord Stevens, who led UK government-commissioned investigations into collusion in Northern Ireland between the 1980s and 2000s, once said that of 210 people arrested during his enquiries, only three were not state agents.

Collusion occurred in a number of ways, he said, ranging from the state withholding intelligence and evidence from investigators, to state agents being involved in murder. In some cases agents told the authorities in advance about planned attacks but nothing was done to stop them.

One of the most notorious agents was codenamed Stakeknife.

It has been publicly known for more than 20 years that Stakeknife was a man called Freddie Scappaticci. He died under MI5 protection in 2023 having never been charged with any of the many murders and other crimes to which he is linked.

Scappaticci was a senior figure in the IRA’s internal security unit, which was tasked by IRA leaders with identifying and killing British agents in the organisation.

But, with the spycatcher himself a spy, the result was one British agent killing other agents, or people accused of being agents, and the state repeatedly not intervening to save them.

A long-running police investigation into Stakeknife, Operation Kenova, was forbidden from officially naming Scappaticci in its final report, published in December, despite the fact that his identity was widely known.

The head of the Kenova team, Sir Iain Livingstone, pleaded with Northern Ireland Secretary Hilary Benn for the right to name Scappaticci in his report, but Benn said he would decide after the Supreme Court ruling in the Paul Thompson case, which came later in December.

That ruling has, in effect, strengthened Benn’s power to stop him being named. If permission is granted, a revised version of the Kenova report can be published.

Jon Boutcher, who was Livingstone’s predecessor leading the Kenova investigation, has described the situation preventing Scappaticci being named as a “pantomime”. It leads to the truth being “suppressed,” he argues, with “profound and severe adverse impacts on victims and families, legitimate public discussion and debate, media freedom, open justice and public confidence in state agencies and the criminal justice system”.

Boutcher, the PSNI chief constable, is involved in a significant and extraordinary stand-off with the government and MI5 over NCND.

He has called for reform of the policy, a principle he fully supports but believes is being wrongly applied and has become a dogma that can be used to conceal wrongdoing.

He has recommended the policy is reviewed to ensure it is applied proportionately and only where genuinely necessary for the protection of life or national security. The government has not yet responded to the call for a review of the policy.

Boutcher has directly asked whether NCND’s continued application on Stakeknife amounts to it being used as a tool for covering up things that should never have happened.

“I question whether this decision protects agents, or rather does it actually protect the government and intelligence agencies from accountability for historical wrongdoing.”

In the Paul Thompson case he argued that he, as chief constable, rather than the government, had the right to allow the coroner to release a summary of police files. The high court and court of appeal backed his position before it was ultimately rejected by the Supreme Court.

During the hearings, government barrister Sir James Eadie KC said Boutcher was being “thoroughly disingenuous” and that “politics is being played with national security”.

He argued the government is best placed to make assessments on national security, as only it has access to the full range of necessary advice. The principal advice, in this context, is from MI5.

The argument the government is best informed seemed somewhat at odds with the fact the MI5 officer whose sworn statement was provided to support the case had not read the relevant underlying material.

For the chief constable of Northern Ireland to be at loggerheads with MI5 is “unprecedented” says veteran investigative journalist, John Ware, who has reported on the Troubles since the 1970s.

“If anybody had told me even 10 years ago that there would come a time when the chief constable of the PSNI and the Security Service were at loggerheads, I would have fallen off my stool,” he says.

Boutcher has a raised a stark question: can the authorities who played a role in the events, including MI5, be trusted to decide whether to tell the public about what happened, when doing so could leave them exposed?

He has also indirectly raised this question about the legal system itself – due to its own role in the Stakeknife case.

Scappaticci helped create a legal precedent on state secrecy that is routinely cited by the government to this day.

In 2003, he sought a judicial review over the then government’s refusal to publicly deny media reports that he was Stakeknife.

But Scappaticci was bluffing and did not actually want to succeed, since if the government was ordered to tell the truth, he would have been confirmed as Stakeknife. Scappaticci was engaged in a pretence to help shore up his reputation: publicly fighting a legal case, while hoping – and expecting – he would lose.

However, the government feared Scappaticci would win because a judge would assume he was not Stakeknife and force ministers to be honest.

Jon Boutcher has now confirmed an extraordinary fact: the judge who heard the case did so after being secretly briefed on the real identity of Stakeknife.

That judge, the then Lord Chief Justice of Northern Ireland, Lord Carswell, ultimately allowed the government to maintain NCND, creating the legal precedent on the policy.

“The full facts about this and the government’s explanation will never be known while it is able to hide behind the shield of NCND,” Boutcher has said.

There is a second extraordinary fact: the government barrister who did the secret briefing was Philip Sales, who as Lord Sales was one of the Supreme Court justices in the Paul Thompson case. This fact was uncovered by the journalist John Ware.

This is an issue because it is Lord Sales’ ruling in the Thompson case which has a central bearing over whether Scappaticci can be named and more details revealed about what went on, including Lord Sales’ own role.

I asked the Supreme Court whether, given Lord Sales’ role in the Scappaticci matter, if it was appropriate for him to have heard the Thompson case.

In a statement, the court said: “Great care is taken in the selection of the panels which will hear a case at the UK Supreme Court.

“A Justice will not deal with a case where it is considered there is a conflict of interest and individual Justices are careful to ensure that they have none.”

Two further cases that touch on the potential release of information about the role of the state during the Troubles are also before the Supreme Court.

Troubles-related inquests were shut down in 2024 with the passing of the Legacy Act, which created a new body called the Independent Commission for Reconciliation and Information Recovery (ICRIR) to examine relevant cases, including those being investigated by police.

The families of four men who were either killed or injured during the Troubles won a further legal victory, when the Northern Ireland Court of Appeal ruled the government has too much veto power over the disclosure of material by the ICRIR.

Lady Chief Justice Siobhan Keegan said that the provision in the Act giving the Northern Ireland secretary “the final say” on disclosure of sensitive national security information by the ICRIR would risk undermining public confidence in the body.

The UK government’s appeal will be ruled on by the Supreme Court in the coming months. The lead respondent in the case is Martina Dillon, whose husband Seamus was killed in 1997.

The paramilitary Loyalist Volunteer Force (LVF) claimed responsibility.

A coroner had opened an inquest on the basis there was evidence of collusion by state authorities, but the inquest was halted by the Legacy Act. No-one has ever been held accountable.

The government has also registered an appeal in another key case.

Sean Brown was also murdered in 1997 by members of the LVF. The 61-year-old father-of-six was abducted as he locked the gates at a sports club in County Londonderry.

There were investigative failures by police and no-one has been brought to justice.

Decades later, a coroner issued a summary of secret material stating that “a number of the individuals linked through intelligence to the murder were agents of the state.” Sean’s family want a public inquiry, and have won judgments over the past two years at the High Court and Court of Appeal.

The government, which was found to have acted unlawfully by refusing a public inquiry, has registered an appeal to the Supreme Court on the basis the issue has constitutional significance regarding who should order inquiries.

At its heart, this vast battle over secrecy centres on two core claims by the state: that only the government and MI5 can ultimately be fully trusted on national security, and that their decisions on what information to release are solely about protecting the public.

But these claims have been undermined by MI5’s actions and omissions in several recent cases, all during the leadership of MI5’s current director general, Sir Ken McCallum.

A public inquiry into the 2017 Manchester Arena bombing, which killed 22 people, concluded that MI5 had not given an “accurate picture” of the key intelligence it held about the suicide bomber who carried out the attack.

The inquiry chairman only uncovered what MI5 had done through his own persistent investigations during secret evidence sessions.

The government has been resisting calls by families bereaved by the Manchester Arena attack for MI5 officers to be fully included in planned new legislation – known as the Hillsborough Law – that’s designed to stop cover-ups and enforce truth-telling by public officials.

Another ongoing case is an example of how MI5’s evidence cannot always be taken at face value: MI5 gave false evidence to three courts about having maintained its NCND secrecy policy in relation to an abusive neo-Nazi agent who attacked his girlfriend with a machete. But MI5 had in fact told me the man was an agent when attempting to prevent me from investigating him.

After MI5 apologised, we then showed that inquiries into what happened by MI5 and the government were deficient and unreliable, with further falsehoods provided to the court. This prompted the most senior judge in England and Wales to dismiss MI5’s explanations and led the prime minister to order a new investigation, which will report this year.

In the Stakeknife investigation, MI5 disclosed key documents years after enquiries began, aer Scappaticci had died and charging decisions had been made, and at a point when the Legacy Act banned police from investigating further.

The new material revealed MI5 had greater and earlier knowledge of Stakeknife than detectives had previously been told, including that MI5 had been involved in tasking him. The lead detective said it was a “significant failure on the part of MI5”.

A review commissioned by MI5 found no deliberate wrongdoing.

Cover-ups and failures to disclose information have a long history in Northern Ireland. Lord Stevens, recalling his investigations, has said: “I was misled deliberately, I was criminally obstructed from doing my job by the Royal Ulster Constabulary and military, whilst MI5 failed to disclose information.”

Judge Pomerance, who was senior counsel on another major investigation, concluded bluntly: “The state viewed itself as above the law”.

Last July, days before Eugene Thompson died, Jon Boutcher delivered a letter of apology on behalf of PSNI. He apologised for the police’s failure to take action that may have prevented Paul Thompson’s murder, and for failures in the investigation, including not properly pursuing suspects.

Mark Thompson from Relatives for Justice (no relation of Paul and Eugene Thompson) says the apology was “important” but that Eugene was by this stage “very ill”.

Mark had watched the Supreme Court hearings with Eugene on a laptop in his hospice and describes the experience as “tough”. Eugene was “praying and hoping that before he died the judgment could be made”, Mark says.

Bereaved families caught up in these cases have had to battle for decades, trying to fill voids left by the lack of proper investigations by the state.

“You became an investigator, tracking down witnesses. You became an expert in the law in that field. And you fought your own case tooth and nail.”

Eugene Thompson became an “accidental lawyer and expert in his own case”, he says, as he campaigned for answers about why his brother Paul was murdered.

“I think if we take anything from the legacy of him, it’s the example of never giving up.”

If you have information about this story or a similar one that you would like to share with Daniel De Simone and the BBC News Investigations team please get in touch. You can contact us in the following ways:

Email: security.investigations@bbc.co.uk

Signal: @investigation.56

Click here to learn how to use SecureDrop, an anonymous whistleblowing tool that works only in the Tor browser and follow the advice to stay secure.

BBC InDepth is the home on the website and app for the best analysis, with fresh perspectives that challenge assumptions and deep reporting on the biggest issues of the day. You can now sign up for notifications that will alert you whenever an InDepth story is published – click here to find out how